Jean  Christofferson 


Monday  1/30/95 


Operations  Status 
Systems  Stability 

wais.com  still  experiencing  crashes  -  but  not  hanging 
Performance  problems  -  need  plan,  resources  to  address. 

internal  Operational  Issues 
Currently  not  being  addressed. 

There  is  a  growing  frustration  internally  with  system  and  network 
performance,  and  disk  space  problems. 

Backup  status  -  haven't  had  good  backup  in  a  week  -  need  quick  solution. 
External  Operational  Issues 

Automatic  monitoring  working  well,  improving  as  incidents  occur. 
CMP  feels  good  but  needs  own  URL  to  be  returned 
Intel  affected  by  internet  problems  not  under  WAIS'  control 
DowVision  status? 

Production  isolation 

Not  in  effect  -  recommend  'demo'  machine. 

Operations  Manual 

Not  complete,  although  portions  working.  Need  time  to  devote  to. 
Staffing 

Dan's  knowledge  not  yet  documented. 

Recommend  develop  increased  expertise  in  system  and  network  issues. 
Jean  -  full  time  hours  required,  20  hrs/week  not  sufficient. 

Operations  Incident  Reports 

DowVision 

wais.com  -  2  crashes 

Move 

Need  determination  if  WAIS  wants  showcase  computer  facility  for  service 
operations. 

Move  is  a  good  opportunity  to  clean  up  computer  room,  internal  network, 
do  internal  network  &  phone  wiring  correctly. 

Recommend  installation  of  'real'  UPS  and  raised  floor  computer  room. 
Need  a  budget  for  move  and  facilities  upgrade. 

Need  manager  devoted  to  move  and  facilities  installation  -  it  is  a  significant 
time  commitment. 


WAIS  Inc.  Service  Operations  1/30/95 
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Notes 

1.)  Do  wais.com  stats  include  C 

MP  Techweb? 

2.)  Would  like  to  add  WAIS  search  stats  in  addition  to  WWW  stats  - 

how  many  users 

how  many  users  do  WAIS  searches?  Could  be  selling  point  over  services  offered  by  others 

3.)  Need  to  get  stats  for  DowVision 

4.)  Need  to  know  where  to  obtain  total  number  of  users,  and  number  of  new  users  for  Scholastic 

Confidential 


JLC 


Operations  Incident  Report 

WAIS,Inc. 


Service: 

Date: 

Time: 

Reported  by: 


CMP  -  TechWeb 
January  23, 1995 
17:54  - 18:01  PST 


WAIS  automatic  monitor 


Description: 

A  system  crash  occurred  on  wais.com  on  Monday  January  23, 1995  at  17:54 
PST.  Diagnostic  information  was  automatically  collected.  Wais.com 
automatically  rebooted  and  was  back  up  at  18:01  PST. 

A  call  has  been  opened  with  Sun  to  determine  cause  of  crash. 
Impact: 

Service  unavailable  from  17:54  until  18:01. 
Action  Items: 

1.)  Reintroducing  nightly  6PM  PST  reboot  until  cause  of  crash  is  identified. 


Operations  Incident  Report 
WAIS,Inc 


Internal  Distribution 


Service: 

Date: 

Time: 

Reported  by: 


DowVision 

Monday  January  23, 1995 
23:00  PST 

Dan  Aronson,  WAIS  automatic  monitor 


Description: 

DowVision  indexing  had  problems  with  locking,  generating  excessive  error 
messages,  resulting  in  filling  disk  space  on  dowvision.  The  automatic 
monitor  detected  the  situation,  but  email  to  wais.airnote.net  was  not  sent  due 
to  disk  space  full  on  sushi,  where  the  dowvision  monitor  runs.  Service  was 
unavailable  until  10:00am  Tuesday  January  24,  1995. 

Data  was  not  indexed  from  14:00  PST  Tuesday  January  24,  until  14:00  PST 
Wednesday  January  25, 1995  while  problem  diagnosed. 

Code  corrected  to  avoid  generation  of  excessive  error  messages.  Indexing 
restored  by  13:00  PST  Thursday  January  26, 1995. 


Service  unavailable  from  23:00  PST  Monday  January  23  until  10:00  PST 
Tuesday  January  24, 1995. 

Data  was  not  indexed  from  14:00  PST  Tuesday  January  24,  until  14:00  PST 
Wednesday  January  25,  1995. 

Action  Items: 

1.)    Investigate  monitoring  confirmation  messages  from  wais.airnote.net  to 
detect  if  a  page  has  not  completed  and  then  use  modem  to  dial  airnote 
directly. 


Impact: 


2.)  Determine  dowvision  operations  status. 


Operations  Incident  Report 
WAIS,  Inc. 


Internal  Distribution 


Service: 

Date: 

Time: 

Reported  by: 


CMP  -  TechWeb,  other  on  wais.com 
Saturday  January  28, 1995 
22:52  PST 
Dan  Aronson 


Description: 

A  system  crash  occurred  on  wais.com  on  Saturday  January  28, 1995  at  22:  52 
PST,  approximently  5  hours  after  rebooting  wais.com.  Diagnostic 
information  was  automatically  collected.  Wais.com  automatically  rebooted 
and  all  services  backup.  Next  pass  of  automatic  monitor  did  not  detect  any 
interruption  in  service. 


Service  unavailable  for  a  short  period  of  time  at  22:52  PST. 
Action  Items: 

1.  )  Forward  core  dump  to  existing  open  call  with  Sun. 

2.  )  Ask  for  additional  resources  within  Sun  to  assist  WAIS  in  resolving  this 
situation.  This  crash  seems  to  indicate  that  rebooting  nightly  does  not 
circumvent  the  error  situation  that  causes  the  crash. 


Impact: 


